Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-20.2: opt: generate lookup joins on partial indexes #54362

Merged

Conversation

mgartner
Copy link
Collaborator

Backport:

Please see individual PRs for details.

/cc @cockroachdb/release

@mgartner mgartner requested a review from a team as a code owner September 14, 2020 20:06
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@mgartner mgartner changed the title release-20.2: TODO release-20.2: opt: generate lookup joins on partial indexes Sep 14, 2020
Copy link
Collaborator

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: assuming it's approved in #53662

Reviewed 4 of 4 files at r1, 3 of 3 files at r2, 5 of 5 files at r3.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @RaduBerinde)

@rytaft
Copy link
Collaborator

rytaft commented Sep 14, 2020

Oops... #53662

Lookup joins are now generated when the ON filters of a join or the
right-side Select filters imply the partial index predicate.

In several cases, the filters are reduced once implication is proven.
This currently prevents optimal plans from being generated in several
cases, either because unnecessary joins remaining in the expression
tree, or because lookup semi- and anti-joins that should be created are
not. I've left TODOs to denote these cases.

Release note (performance improvement): The optimizer can now use
partial indexes for lookup join operations. This results in potentially
more efficient query plans for joins on tables with partial indexes.
This commit updates `GenerateLookupJoins` to generate lookup semi- and
anti-joins in cases where the columns in the original ON filters are not
covered by the index, but the columns in the filters remaining after
proving filter-predicate implication are covered by the index. This is a
special case only possible for semi- and anti-joins because they never
include columns from the right side of the join in their output columns.
See the comment added in `GenerateLookupJoins` for more details.

Release note (performance improvement): Lookup semi- and anti-joins are
now explored by the optimizer in more cases when the ON filter implies
a partial index predicate. This may lead to more efficient query plans
in some cases.
Previously, when generating a special case of lookup semi/anti-joins on
partial indexes, all columns from the right side of the input join would
be included in the lookup join's columns. These columns were included
even though they were no longer needed (in this special case, the ON
filters have been reduced during filter/predicate implication).

Fetching unnecessary columns was was causing a slight difference between
the estimated cost of a lookup semi/anti-join on a non-partial index
versus a partial index. This is because the coster adds a slight
overhead for each column looked-up by the lookup join.

This commit correctly assigns the lookup join columns in this special
case so that unnecessary columns are not included.

In addition, an assertion has been added to `check_expr.go` (run during
`make testrace`) to catch future mistakes like this. Some tests that
manually build expressions had to be edited to avoid violating this
assertion.

Release note: None
@mgartner mgartner force-pushed the backport20.2-54157-54217-54281 branch from 0bd5664 to 2470946 Compare September 16, 2020 20:29
@mgartner mgartner merged commit 1305726 into cockroachdb:release-20.2 Sep 16, 2020
@mgartner mgartner deleted the backport20.2-54157-54217-54281 branch September 16, 2020 21:12
@rafiss rafiss added this to the 20.2 milestone Apr 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants